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Abstract 

Introduction: Community-dwelling stroke survivors tend to become less physically active over time. There is no 
'gold standard' to measure walking activity in this population. Assessment of walking activity generally involves 
subjective or observer-rated instruments. Objective measuring with an activity monitor, however, gives more insight 
into actual walking activity. Although several activity monitors have been used in stroke patients, none of these 
include feedback about the actual walking activity. FESTA (FEedback to Stimulate Activity) determines number of 
steps, number of walking bouts, covered distance and ambulatory activity profiles over time and also provides 
feedback about the walking activity to the user and the therapist. 

Aim: To examine the criterion validity and test-retest-reliability of the FESTA as a measure of walking activity in 
patients with chronic stroke. To target the properties of the measurement device itself and thus exclude effects of 
behavioral variability as much as possible evaluation was performed in standardized activities. 

Methods: Community-dwelling individuals with chronic stroke were tested twice with a test-retest interval varying 
from two days to two weeks. They performed a six-minute walk test and a standardized treadmill test at different 
speeds on both testing days. Walking activity was expressed in gait parameters: steps, mean-step-length and walking 
distance. Output data of the FESTA on the treadmill was compared with video analysis as the criterion measurement. 
Intraclass Correlations Coefficients (ICCs) and Mean Relative Root Squared Error (MRRSE) were calculated. 

Results: Thirty-three patients were tested to determine criterion validity, 27 patients of this group were tested twice for 
test-retest reliability. ICC values for validity and reliability were high, ranging from .841 to .972. 

Conclusion: This study demonstrated good criterion validity and test-retest-reliability of FESTA for measuring specific 
gait parameters in chronic stroke patients. FESTA is a valid and reliable tool for capturing walking activity measurements 
in stroke, and has applicability to both clinical practice and research. 
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Introduction 

In many Western nations, stroke is a leading cause of 
death and serious long-term disability [1]. A frequent 
consequence of stroke is unilateral loss or limitation of 
muscle function, leading to a loss of mobility, movement 
and functional ability [2,3]. Van de Port et al (2006) 
showed that a substantial proportion of community- 
dwelling stroke survivors becomes less physically active 
over time [4]. Post-stroke physical inactivity may produce 
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physical deconditioning, and as a consequence a decline 
in function [5]. A decline in function reduces participation 
in the community and quality of life [6] and decreases 
independence of the stroke survivor [5]. Furthermore, 
physical inactivity increases the risk of developing co- 
morbidities and having a recurrent stroke [5]. Accurate 
measurement of real life walking activity could be 
beneficial in tailoring rehabilitation. Using actual per- 
formance data and providing feedback might support 
self-management strategies to prevent physical and func- 
tional decline and subsequent consequences. 

Currently assessment of walking activity generally in- 
volves subjective or observer- rated instruments [7]. 
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These instruments have disadvantages such as the risk 
of recall bias, social desirability of answers, and poor 
generalisation [7]. Objective assessment of the number 
of steps can be done with pedometers. Roos etal (2012) 
demonstrated the disadvantage of measuring only the 
total number of steps taken [8]. They found differences 
in walking bouts and walking time between older adults 
and stroke survivors and that it varied based on func- 
tional ability. This relevant variation could not have been 
identified when measuring only steps per day [8]. Meas- 
uring gait parameters with accelerometers overcomes 
the limitation of measuring only the number of steps. 
To measure gait parameters by accelerometry in this 
population specific algorithms are required since stroke 
survivors are slow walkers [9] and accuracy of detecting 
steps decreases when gait speed and step frequency de- 
crease [10]. To date, several motion sensors have been 
used [11,12], such as the accelerometer based Step- 
Watch Activity Monitor (SAM) which has good validity 
in measuring gait parameters in stroke survivors. How- 
ever, current devices are not capable of providing feed- 
back to the stroke survivor about their walking activity. 
Providing feedback about their walking activity might 
prevent physical inactivity, and as a consequence a de- 
cline in function [5]. To monitor walking and to investi- 
gate potential beneficial effects of feedback in stroke 
survivors we developed FESTA. 

FESTA (FEedback to STimulate Activity) is a telemet- 
ric system that includes a tri-axial piezo capacitive accel- 
erometer which can be coupled to a docking station. 
The docking station is capable of; calculating gait pa- 
rameters, evaluating whether the amount of walking ac- 
tivity during the day was sufficient according to the goal 
set by the physical therapist, providing the feedback to a 
screen visible for the stroke patient, sending an email to- 
wards the physical therapist with the calculated gait pa- 
rameters and recharging the battery of the accelerometer 
to continue monitoring the next day. 

As measuring gait parameters is more challenging in 
stroke survivors, the first step in this developing process 
was to examine the criterion validity and test-retest reli- 
ability of FESTA at gait parameter recognition in chronic 
stroke survivors using a stroke specific developed algo- 
rithm. We examined gait parameters; number of steps, 
mean-step-length with a standardized treadmill test and 
walking distance with an over ground 6 minute walk 
test. Furthermore FESTA calculates walking time and 
walking bouts as a derivative from steps [13]. 

Methods 

Participants 

A convenience sample of community-dwelling, chronic 
stroke survivors was recruited from ten private physical 
therapy practices, the daycare center of 'Zorgspectrum' 



and the patients' association 'Samen verder' in the 
Netherlands and the University of Maryland in the 
United States of America. Stroke was defined according to 
the World Health Organization definition. Participants 
were able to walk independently without physical assist- 
ance (Functional Ambulation Categories score > 3) [14] and 
were at least three months post stroke.Participants were ex- 
cluded if they had severe cognitive disorders (Mini-Mental 
State Examination <24) [15], severe communicative dis- 
orders (Utrechts Communicatie Onderzoek <4) [16] or 
acute disorders impairing gait. All participants gave written 
informed consent prior to participation in the study. 
The research protocol and all informational material were 
approved by the Medical Ethical Committee (MEC) of the 
University Medical Center Utrecht and the Institutional 
Review Board of the University of Maryland, Baltimore. 
Treatment of the participants was according to the Helsinki 
declaration [17]. 

Equipment & experimental protocol 
Procedure 

Participants were tested twice with a test-retest interval 
of a minimum of two days and a maximum of two weeks 
using the six-minute walk test (6MWT) and a standard- 
ized treadmill test. At baseline, inclusion measurements 
and collection of personal and anthropometric data were 
performed prior to the physical tests. 

FESTA monitor 

During both tests the FESTA was worn around the back 
site of the waist, between the posterior superior iliac 
spines. The FESTA contains one tri-axial, piezo-capacitive 
accelerometer (70*80*25 mm, 150 grams, range ±2.5 g). 
Based on sensor alignment, acceleration signals were iden- 
tified as anterior-posterior (AP), medio-lateral (ML) and 
vertical (VT). Output is in mV, a change of 1 mV corre- 
sponded to a change of 0.08 m/s^ (resolution)). Acceler- 
ation signals were digitally stored on a memory card with 
a sampling rate of 25 samples/s. 

6MWT 

The 6MWT was performed to assess overground walk- 
ing distance. The 6MWT was performed according to 
the American Thoracic Society Guidelines [18]. Walked 
distance was determined by counting the number of 
walked laps (20 meters) and adding any final fraction of 
laps, measured by a measuring wheel. Results were used 
to calculate the comfortable walking speeds for the 
treadmill test (CWT) and to assess the overground walk- 
ing distance validity and reliability of the FESTA. 

Standardized treadmill test 

Gait parameters, number of steps and mean-step-length 
were determined using a standardized treadmill test. 
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Because accuracy of the gait parameter; steps recogni- 
tion depends on gait speed and gait speed may vary dur- 
ing a day and is low in this population [9] we executed a 
treadmill test at three different gait speeds within each 
subject. Gait speeds were established at 15% below, 
equally to and 15% above comfortable gait speed. Each 
speed condition lasted for two minutes. The mean walk- 
ing speed measured by the 6MWT -10% was used to 
define the comfortable walking speed. Fingertip handrail 
support was allowed during testing. The treadmills (En 
Mill treadmill, Enraf Nonius, the Netherlands and Gait 
Trainer 3™, Biodex, USA) were calibrated prior to the 
study. A camera was placed 1.2 meter behind the tread- 
mill (Panasonic type HC-V70, 50 samples/s). 

Data processing and algorithms 

From every block of two minutes at different speeds, 
only the last 90 seconds were analysed. The researcher 
counted the number of steps during these blocks of 
90 seconds from the video afterwards and was blinded 
from the results of FESTA. Distances from the treadmill 
test were determined by using the treadmill speed and 
the testing time of the treadmill test. The mean-step- 
length was derived from the distance walked and divided 
by the steps taken by both legs. 

From the same blocks of 90 seconds, the gait parame- 
ters (number of steps and mean-step-length) from 
FESTA were analysed using Matlab (Matlab 7.10.1, The 
Math Works Inc, US A). For the step detection we used 
spectral analysis derived from the AP acceleration signal. 
Taking the individual variety of the distance-acceleration 
relationship into account, we used an individual calibra- 
tion procedure for distance measures to determine the 
acceleration-distance relation [19]. Firstly we calculated 
the root mean square of the AP acceleration signal, sec- 
ondly conducted a linear fit (first order polynomial) be- 
tween the different gait speeds and the different root 
mean square values, thirdly we used the polynomial 
function to predict the walking speed and subsequently 
walking distance in the treadmill test and 6MWT.This 
distance prediction derived from a single acceleration 
signal and the individual calibration procedure is de- 
scribed by Schutz et al. (2002) [19] in more detail. 

To assess the validity and reliability of FESTA, we 
compared the gait parameters derived from FESTA with 
the gold standard (video analysis). Comparisons for the 
gait parameters number of steps and mean-step-length 
were performed by using the data from the treadmill 
test. This procedure is consistent with procedures from 
similar validation studies [20-23] and video analysis 
seems to be the most appropriate criterion standard for 
the assessment of physical activity [23]. The comparison 
of the gait parameter walking distance was performed 
using the data derived from the 6MWT. 



Statistics 

Descriptive statistics were performed for all variables 
and normality was assessed by visual inspection of histo- 
grams and quantile-quantile plots. An ICCsj of > .75 
was defined as high as suggested by Burdock et al.(1963) 
[24]. All calculations were performed using SPSS (IBM 
Software, SPSS Statistics 20, USA) or Madab (Matlab 
7.10.1, The MathWorks Inc, USA). 

Validity 

To assess the level of agreement between FESTA and 
the gold standard, and thus the criterion validity, single 
measures intraclass correlation coefficientSagreement (ICCs^i, 
Two-way mixed model) were calculated for the different 
gait parameters; number of steps and mean-step-length 
obtained from the treadmill test and over ground 
walked distance in the 6MWT. Furthermore the Mean 
Relative Root Square Error (MRRSE) was calculated for 
each parameter. The MRRSE is a measure of the differ- 
ences between the values of FESTA and the observed 
values, relative to the unit of measurement (see Formula). 
The MRRSE gives an indication of the mean error of 
FESTA per step or number of steps as a percentage of the 
measurement unit. 



/y^^ (Xobs, i-X models iV 
MRRSE = \\^^^ * 100 

V ^ohs 

^obs = mean of the observed values, criterion measure- 
ment, video analysis 
^mo deli = values obtained by FESTA 

Reliability 

Single measures intraclass correlation coefficientSconsistency 
(ICCs,!, Two-way mixed model) was calculated to analyse 
the test-retest reliability of FESTA. Additionally, the Min- 
imal Detectable Change (MDC95) was calculated from 
the Standard Error of Measurement (SEM) as MDC95 = 
[1.96*SEMconsistency *V2] and SEM = [sd * ^/\T^. where r 
is the test-retest reliability coefficient ICCagreement 3,1 and 
sd is the standard deviation of the scores at the first test 
occasion (TO). The SEM is multiplied by 1.96 to deter- 
mine the 95% confidence interval and multiplied by the 
square root of 2 to account for the additional error associ- 
ated with repeated measurements [25]. The MDC95 is the 
minimal amount of change that must be observed before 
the change can be considered to exceed the variation and 
measurement error at the 95% confidence level. 

Results 

A total of 33 participants (17 men and 16 women) were 
tested and their data were used to determine the criter- 
ion validity of FESTA. Twenty-seven participants were 



Punt et al. Journal of NeuroEngineering and Rehabilitation 2014, 11:30 
http://www.jneuroengrehab.conn/content/1 1/1/30 



Page 4 of 7 



Table 1 Validity results FESTA 



Speed 



Parameter 



Video analysis 



FESTA 



MRRSE (%) 



ICC 



Speed 1 =CWT- 15% 
Mean±SD:2.4±l.l km/h 
Range: 0.3 - 4.4 km/h 

Speed 2 = CWT 

Mean±SD: 2.8 ±1.2 km/h 
Range: 0.4 - 5.2 km/h 

Speed 3 = CW+15% 
Mean±SD: 3.2 ±1.4 km/h 
Range: 0.5 - 5.6 km/h 



Step count (steps) 
Mean step length (m) 

Step count (steps) 
Mean step length (m) 

Step count (steps) 
Mean step length (m) 



Mean ± SD 
129±25 
0.45 ±0.14 

138±27 
0.50 ±0.1 5 

145 ±28 
0.54 ±0.1 7 



Mean ± SD 

135±21 
0.43 ±0.1 6 

141 ±23 
0.48 ±0.1 6 

146 ±25 
0.52 ±0.1 8 



5.8 
9.1 

3.5 
6.2 

3.4 
5.3 



,841 
,910 

,964 
,964 

,964 
,971 



MRRSE = Mean Relative Root Squared Error; percentage mean absolute deviation, ICC = Intraclass Correlation Coefficient, CWT = Comfortable Walking Speed 
for Treadmill. 



tested twice. The other six participants did not perform 
a second test, due to motivational problems to perform 
a second test or being unable to perform a second test 
within the set time limit of two weeks after the first test. 
The mean age of the 33 participants was 61.8 ± 8.8 years, 
time since stroke was 5.6 years ± 3.8 years and the func- 
tional ambulation category (FAC) scores ranged from 3 
to 5 (mean 4.4 ± 0.7). The average distance walked in the 
6MWT was 317.3 meters, which is 0.88 m/s, ranging 
from 36 to 580 meters. For the treadmill testing, the dif- 
ferent walking speeds varied from 0.08 to 1.5 m/s. 

Validity 

For number of steps and mean-step-length at the three 
different gait speeds, ICCagreement 3,1 varied between 
0.841 and 0.971 (p < 0.001 for all values). Mean Relative 
Root Squared Errors (MRRSE) ranged between 3.4 
and 9.1%. All agreement parameters are presented in 
Table 1. 

Figure 1 illustrates the differences between the gold 
standard and FESTA for the gait parameters steps and 
mean-step-length, with the difference in steps (top 
panel) and mean-step-length (bottom panel). 



Criterion validity for overground walking distance dur- 
ing the 6MWT is presented in Table 2. The difference 
between measured and estimated over ground walking 
distance in meters averaged -20.1 meters see Figure 2. 

Reliability 

Table 3 presents the test-retest reliability for the gait pa- 
rameters steps and mean-step-length, including ICC 
values and MDC. ICCconsistency 3,1 scores ranged from 
0.876 to 0.972 and were all significant at p < 0.001. 

Figure 3 illustrates the differences between the first and 
second test occasion, reliability of the gait parameters; 
steps (top panel) and mean-step-length (bottom panel). 

Test-retest reliability for over ground distance covered 
during the 6MWT for ICCagreement 3,1 >was .97. Mean dif- 
ference in meters was 8.1 meter, see Figure 4. 

Discussion 

The objective of this study was to examine the criterion 
validity and test-retest reliability of the novel telemetric 
system, FESTA, in measuring walking activity in stroke 
survivors. To this end, we tested gait parameters; steps, 
mean-step-length and walking distance in chronic stroke 
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Figure 1 FESTA estimates of steps minus golden standard (top panel). FESTA estimates of mean-step-length minus golden standard (bottom 
panel). At 15% below comfortable walking speed {CWT -15%) equal to {CWT) and 15% above. 
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Table 2 Validity overground walking distance 



Table 3 Test-retest reliability of gait parameters obtained 





6IVIWT (m) 


6MWT (m) 


MRRSE (%) 


ICC 


by FESTA 










measured 


FESTA 






Speed 


Parameter 


TO 


T1 ICC MDC95 


Mean ± SD 


3 1 7.3 ± 134.7 


337.4 ±136.3 


12.1 


.937 






{Mean ± SD) 


{Mean ± SD) 


Range 


36.0 -580.0 


44-581.5 


Speed 1 =CV\^- 15 


% Step count 
(steps) 


136 ±21.0 


135 ±20.4 .938 14.3 



6MWT= Six Minute Walk Test, MMRSE = Mean Relative Root Squared Error; 
percentage mean absolute deviation, ICC = Intraclass Correlation Coefficient. 



survivors. Results of criterion validity and test-retest reli- 
ability indicate good validity and reliability as all ICC 
values were between .841 and .972. These results are 
similar to the most commonly used accelerometer in the 
stroke population [10,12,26]. Moreover, the results present 
higher accuracy in comparison to algorithms not specific- 
ally developed for the stroke population [26]. 

No clear trend can be seen between ICC values and 
MRRSE and the three gait speed conditions. This indi- 
cates that the validity of the FESTA is not affected by 
gait speed. Although the latter finding demonstrates the 
possible robustness of the FESTA for real-life use, we 
have to take into account that gait parameters differ for 
treadmill walking and over ground walking [27]. When 
walking on a treadmill, the gait patterns of chronic 
stroke survivors are more symmetrical and stable com- 
pared to overground walking. Furthermore in real-life 
gait speed may vary during a day and even within a 
walking bout. Therefore the gait parameters steps and 
mean-step-length have to be interpreted with caution 
since these parameters were only tested on the treadmill 
and might not be generalizable to walking overground. 
Further research is needed to determine these outcomes 
in overground walking. Another limitation of the study 
was the test-retest reliability design. Although all condi- 
tions were similar in the first and second test occasion 
and subjects were stable, subjects did perform slightly 
differently in the first and second test. For example, sub- 
jects took slightly fewer steps in the treadmill test and 
walked a few meters further in the second 6MWT 
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Figure 2 FESTA estimates of covered distance (M) minus 
golden standard at the 6N\WT. 



600 



Mean step 0.44±0.15 0.42±0.14 .876 0.14 
length (m) 

Speed 2 = CV\^ Step count 141 ±22.5 140±22.6 .949 141 

(steps) 

Mean step 0.48 ±0.1 4 0.48 ±0.1 5 .942 0.10 
length (m) 

Speed 3 = CV\^+ 15% Step count 145 ±23.9 144 ± 25.6 .972 114 
(steps) 

Mean step 0.52 ±0.1 6 0.53 ±0.1 6 .944 0.10 
length (m) 

CWT = Comfortable Walking speed for Treadmill, ICC = Intraclass Correlation 
Coefficient, MDC = Minimal Detectable Change. 



compared to the first test occasion. This affected the re- 
liability results of FESTA. 

For specific measurement devices, measurement errors 
should be smaller than the Minimal Clinically Important 
Difference (MCID) to detect a valuable clinical effect for 
individuals. For the treadmill tests, no MCIDs have been 
defined. The MDC95 score of the FESTA for overground 
walked distance at the 6MWT was 62.2 meters. Unfortu- 
nately the interpretation of this value remains unclear as 
the MDC of FESTA depends on de MDC of the 6MWT 
since the MDC of FESTA is derived during the 6MWT. 
The MDC of the 6MWT for this population ranges from 
29 m [28] towards 54.1 m [29]. 

Statistical considerations 

Previous studies with a similar design and aim as we had 
[11,12,30,31] expressed accuracy performance in ICC 
values and Limits of Agreement. In this study, we added 
a new measure for validity; the Mean Relative Root 
Squared Error (MRRSE). It is known that ICC values are 
strongly influenced by the magnitude of the variance 
within the study sample. Furthermore, other than the 
name ICCagreement suggests, the ratio of variances is cal- 
culated, rather than the absolute agreement score [32]. 
When taking a closer look at the ICC formula, it is clear 
that a large variance in subject scores, as is the case in 
this study, will lead to a higher ICC [32]. Studies with 
different variances in their study populations can there- 
fore not be compared directly. To get a better insight in 
the true agreement between the output of the FESTA 
and the 'gold standard', and to eliminate the effect of the 
high variance in our study population, we calculated the 
MRRSE for each gait parameter. The MRRSE represents 
the mean absolute percentage difference between the 
two measurement devices, expressed in the percentage 
of unit of the parameter. This score is easy to use in 
daily practice, easy to interpret and not dependent of 
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Figure 3 FESTA estimates, difference between steps at the first and second test occasion (top panel). FESTA estimates, difference 
between mean-step-lengtli at tine first and second test occasion (bottom panel). 
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variance between patients. Therefore, we hereby suggest 
using the MRRSE in future research, as it provides a 
more direct comparison between studies and between 
measurement devices. 

FESTA 

FESTA (FEedback to STimulate Activity) is a newly de- 
veloped telemetric system and validity and reliability 
were shown to be good in the present analysis. It is de- 
signed to monitor and stimulate stroke survivors with 
respect to their daily walking activity. The physical ther- 
apist is able to interact by setting walking activity goals 
based on walking time and walking bouts. FESTA has 
several advantages over other methods for assessing 
walking activity; it can measure different gait parameters 
such as number of steps, mean-step-length, distance and 
as derivatives walking time and the number of walking 
bouts [13], whereas a step-counter can only determine 
the number of steps. Roos et. al. (2012) clearly stated 
that steps alone is not sufficient to characterize physical 
inactivity in stroke survivors [8]. Due to the docking sta- 
tion FESTA is not limited by battery life and data cap- 
acity. Therefore it is able to monitor for a long time 
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Figure 4 FESTA estimates difference in covered wallcing 
distance (M) between the first and second test occasion. 
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period without recharging or removing data. Further- 
more FESTA provides the researcher, physical therapist 
and stroke survivor with real-life walking activity infor- 
mation. Future research will involve studying the effect 
of giving feedback using this device. The aim will be to 
increase walking activity by providing feedback to the 
user and providing information of actual walking activity 
and the daily pattern of walking activity to the physical 
therapist. Using FESTA provides new possibilities to 
measure walking activity of chronic stroke survivors in a 
valid and reliable way and thereby offers a variety of per- 
spectives for research and treatment in this population. 

Conclusion 

Based on ICC values and MRRSE, this study demon- 
strated good criterion validity and test-retest reliability 
of the telemetric system FESTA for measuring gait pa- 
rameters in chronic stroke survivors. FESTA provides 
the possibility to measure gait parameters in a valid and 
reliable manner and can be used, in both clinical prac- 
tice and academic research. 
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